{
 "cells": [
  {
   "cell_type": "raw",
   "id": "afaf8039",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "sidebar_label: Qdrant\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49f1e0d",
   "metadata": {},
   "source": [
    "# Qdrant\n",
    "\n",
    "This guide will help you getting started with such a retriever backed by a [Qdrant vector store](/docs/integrations/vectorstores/qdrant). For detailed documentation of all features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain.retrievers_self_query.SelfQueryRetriever.html).\n",
    "\n",
    "## Overview\n",
    "\n",
    "A [self-query retriever](/docs/how_to/self_query/) retrieves documents by dynamically generating metadata filters based on some input query. This allows the retriever to account for underlying document metadata in addition to pure semantic similarity when fetching results.\n",
    "\n",
    "It uses a module called a `Translator` that generates a filter based on information about metadata fields and the query language that a given vector store supports.\n",
    "\n",
    "### Integration details\n",
    "\n",
    "| Backing vector store | Self-host | Cloud offering | Package | [Py support](https://python.langchain.com/docs/integrations/retrievers/self_query/qdrant_self_query/) |\n",
    "| :--- | :--- | :---: | :---: | :---: |\n",
    "[`QdrantVectorStore`](https://api.js.langchain.com/classes/langchain_qdrant.QdrantVectorStore.html) | ✅ | ✅ | [`@langchain/qdrant`](https://www.npmjs.com/package/@langchain/qdrant) | ✅ |\n",
    "\n",
    "## Setup\n",
    "\n",
    "Set up a Qdrant instance as documented [here](/docs/integrations/vectorstores/qdrant). Set the following environment variables:\n",
    "\n",
    "```ts\n",
    "process.env.QDRANT_URL = \"YOUR_QDRANT_URL_HERE\" // for example, http://localhost:6333\n",
    "```\n",
    "\n",
    "If you want to get automated tracing from individual queries, you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
    "\n",
    "```typescript\n",
    "// process.env.LANGSMITH_API_KEY = \"<YOUR API KEY HERE>\";\n",
    "// process.env.LANGSMITH_TRACING = \"true\";\n",
    "```\n",
    "\n",
    "### Installation\n",
    "\n",
    "The vector store lives in the `@langchain/qdrant` package. You'll also need to install the `langchain` and `@langchain/community` packages to import the main `SelfQueryRetriever` classes.\n",
    "\n",
    "For this example, we'll also use OpenAI embeddings, so you'll need to install the `@langchain/openai` package and [obtain an API key](https://platform.openai.com):\n",
    "\n",
    "```{=mdx}\n",
    "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
    "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
    "\n",
    "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
    "\n",
    "<Npm2Yarn>\n",
    "  @langchain/qdrant langchain @langchain/community @langchain/openai @langchain/core\n",
    "</Npm2Yarn>\n",
    "```\n",
    "\n",
    "The official Qdrant SDK (`@qdrant/js-client-rest`) is automatically installed as a dependency of `@langchain/qdrant`, but you may wish to install it independently as well."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a38cde65-254d-4219-a441-068766c0d4b5",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "First, initialize your Qdrant vector store with some documents that contain metadata:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e7fd15a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { OpenAIEmbeddings } from \"@langchain/openai\";\n",
    "import { QdrantVectorStore } from \"@langchain/qdrant\";\n",
    "import { Document } from \"@langchain/core/documents\";\n",
    "import type { AttributeInfo } from \"langchain/chains/query_constructor\";\n",
    "\n",
    "import { QdrantClient } from \"@qdrant/js-client-rest\";\n",
    "\n",
    "/**\n",
    " * First, we create a bunch of documents. You can load your own documents here instead.\n",
    " * Each document has a pageContent and a metadata field. Make sure your metadata matches the AttributeInfo below.\n",
    " */\n",
    "const docs = [\n",
    "  new Document({\n",
    "    pageContent:\n",
    "      \"A bunch of scientists bring back dinosaurs and mayhem breaks loose\",\n",
    "    metadata: { year: 1993, rating: 7.7, genre: \"science fiction\" },\n",
    "  }),\n",
    "  new Document({\n",
    "    pageContent:\n",
    "      \"Leo DiCaprio gets lost in a dream within a dream within a dream within a ...\",\n",
    "    metadata: { year: 2010, director: \"Christopher Nolan\", rating: 8.2 },\n",
    "  }),\n",
    "  new Document({\n",
    "    pageContent:\n",
    "      \"A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea\",\n",
    "    metadata: { year: 2006, director: \"Satoshi Kon\", rating: 8.6 },\n",
    "  }),\n",
    "  new Document({\n",
    "    pageContent:\n",
    "      \"A bunch of normal-sized women are supremely wholesome and some men pine after them\",\n",
    "    metadata: { year: 2019, director: \"Greta Gerwig\", rating: 8.3 },\n",
    "  }),\n",
    "  new Document({\n",
    "    pageContent: \"Toys come alive and have a blast doing so\",\n",
    "    metadata: { year: 1995, genre: \"animated\" },\n",
    "  }),\n",
    "  new Document({\n",
    "    pageContent: \"Three men walk into the Zone, three men walk out of the Zone\",\n",
    "    metadata: {\n",
    "      year: 1979,\n",
    "      director: \"Andrei Tarkovsky\",\n",
    "      genre: \"science fiction\",\n",
    "      rating: 9.9,\n",
    "    },\n",
    "  }),\n",
    "];\n",
    "\n",
    "/**\n",
    " * Next, we define the attributes we want to be able to query on.\n",
    " * in this case, we want to be able to query on the genre, year, director, rating, and length of the movie.\n",
    " * We also provide a description of each attribute and the type of the attribute.\n",
    " * This is used to generate the query prompts.\n",
    " */\n",
    "const attributeInfo: AttributeInfo[] = [\n",
    "  {\n",
    "    name: \"genre\",\n",
    "    description: \"The genre of the movie\",\n",
    "    type: \"string or array of strings\",\n",
    "  },\n",
    "  {\n",
    "    name: \"year\",\n",
    "    description: \"The year the movie was released\",\n",
    "    type: \"number\",\n",
    "  },\n",
    "  {\n",
    "    name: \"director\",\n",
    "    description: \"The director of the movie\",\n",
    "    type: \"string\",\n",
    "  },\n",
    "  {\n",
    "    name: \"rating\",\n",
    "    description: \"The rating of the movie (1-10)\",\n",
    "    type: \"number\",\n",
    "  },\n",
    "  {\n",
    "    name: \"length\",\n",
    "    description: \"The length of the movie in minutes\",\n",
    "    type: \"number\",\n",
    "  },\n",
    "];\n",
    "\n",
    "/**\n",
    " * Next, we instantiate a vector store. This is where we store the embeddings of the documents.\n",
    " * We also need to provide an embeddings object. This is used to embed the documents.\n",
    " */\n",
    "\n",
    "const client = new QdrantClient({ url: process.env.QDRANT_URL });\n",
    "\n",
    "const embeddings = new OpenAIEmbeddings();\n",
    "const vectorStore = await QdrantVectorStore.fromDocuments(docs, embeddings, {\n",
    "  client,\n",
    "  collectionName: \"movie-collection\",\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f321c7bd",
   "metadata": {},
   "source": [
    "Now we can instantiate our retriever:\n",
    "\n",
    "```{=mdx}\n",
    "import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
    "\n",
    "<ChatModelTabs customVarName=\"llm\" />\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "821ea7a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "// @lc-docs-hide-cell\n",
    "\n",
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "\n",
    "const llm = new ChatOpenAI({\n",
    "  model: \"gpt-4o\",\n",
    "  temperature: 0,\n",
    "});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "70cc8e65-2a02-408a-bbc6-8ef649057d82",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { SelfQueryRetriever } from \"langchain/retrievers/self_query\";\n",
    "import { QdrantTranslator } from \"@langchain/community/structured_query/qdrant\";\n",
    "\n",
    "const selfQueryRetriever = SelfQueryRetriever.fromLLM({\n",
    "  llm: llm,\n",
    "  vectorStore: vectorStore,\n",
    "  /** A short summary of what the document contents represent. */\n",
    "  documentContents: \"Brief summary of a movie\",\n",
    "  attributeInfo: attributeInfo,\n",
    "  structuredQueryTranslator: new QdrantTranslator(),\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c5f2839-4020-424e-9fc9-07777eede442",
   "metadata": {},
   "source": [
    "## Usage\n",
    "\n",
    "Now, ask a question that requires some knowledge of the document's metadata to answer. You can see that the retriever will generate the correct result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "51a60dbe-9f2e-4e04-bb62-23968f17164a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  Document {\n",
      "    pageContent: 'A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea',\n",
      "    metadata: { director: 'Satoshi Kon', rating: 8.6, year: 2006 },\n",
      "    id: undefined\n",
      "  },\n",
      "  Document {\n",
      "    pageContent: 'Three men walk into the Zone, three men walk out of the Zone',\n",
      "    metadata: {\n",
      "      director: 'Andrei Tarkovsky',\n",
      "      genre: 'science fiction',\n",
      "      rating: 9.9,\n",
      "      year: 1979\n",
      "    },\n",
      "    id: undefined\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "await selfQueryRetriever.invoke(\n",
    "  \"Which movies are rated higher than 8.5?\"\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfe8aad4-8626-4330-98a9-7ea1ca5d2e0e",
   "metadata": {},
   "source": [
    "## Use within a chain\n",
    "\n",
    "Like other retrievers, Qdrant self-query retrievers can be incorporated into LLM applications via [chains](/docs/how_to/sequence/).\n",
    "\n",
    "Note that because their returned answers can heavily depend on document metadata, we format the retrieved documents differently to include that information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "23e11cc9-abd6-4855-a7eb-799f45ca01ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n",
    "import { RunnablePassthrough, RunnableSequence } from \"@langchain/core/runnables\";\n",
    "import { StringOutputParser } from \"@langchain/core/output_parsers\";\n",
    "\n",
    "import type { Document } from \"@langchain/core/documents\";\n",
    "\n",
    "const prompt = ChatPromptTemplate.fromTemplate(`\n",
    "Answer the question based only on the context provided.\n",
    "\n",
    "Context: {context}\n",
    "\n",
    "Question: {question}`);\n",
    "\n",
    "const formatDocs = (docs: Document[]) => {\n",
    "  return docs.map((doc) => JSON.stringify(doc)).join(\"\\n\\n\");\n",
    "}\n",
    "\n",
    "// See https://js.langchain.com/docs/tutorials/rag\n",
    "const ragChain = RunnableSequence.from([\n",
    "  {\n",
    "    context: selfQueryRetriever.pipe(formatDocs),\n",
    "    question: new RunnablePassthrough(),\n",
    "  },\n",
    "  prompt,\n",
    "  llm,\n",
    "  new StringOutputParser(),\n",
    "]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d47c37dd-5c11-416c-a3b6-bec413cd70e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The movies rated higher than 8.5 are the ones directed by Satoshi Kon (rating: 8.6) and Andrei Tarkovsky (rating: 9.9).\n"
     ]
    }
   ],
   "source": [
    "await ragChain.invoke(\"Which movies are rated higher than 8.5?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c52ef888",
   "metadata": {},
   "source": [
    "## Default search params\n",
    "\n",
    "You can also pass a `searchParams` field into the above method that provides default filters applied in addition to any generated query. The filter syntax is the same as the backing Qdrant vector store:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f6103afe",
   "metadata": {},
   "outputs": [],
   "source": [
    "const selfQueryRetrieverWithDefaultParams = SelfQueryRetriever.fromLLM({\n",
    "  llm: llm,\n",
    "  vectorStore: vectorStore,\n",
    "  documentContents: \"Brief summary of a movie\",\n",
    "  attributeInfo: attributeInfo,\n",
    "  structuredQueryTranslator: new QdrantTranslator(),\n",
    "  searchParams: {\n",
    "    filter: {\n",
    "      must: [\n",
    "        {\n",
    "          key: \"metadata.rating\",\n",
    "          range: {\n",
    "            gt: 8.5,\n",
    "          },\n",
    "        },\n",
    "      ],\n",
    "    },\n",
    "    mergeFiltersOperator: \"and\",\n",
    "  },\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",
    "For detailed documentation of all Qdrant self-query retriever features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain.retrievers_self_query.SelfQueryRetriever.html)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "TypeScript",
   "language": "typescript",
   "name": "tslab"
  },
  "language_info": {
   "codemirror_mode": {
    "mode": "typescript",
    "name": "javascript",
    "typescript": true
   },
   "file_extension": ".ts",
   "mimetype": "text/typescript",
   "name": "typescript",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
